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am  just  going  to  say  it:  I  don't 
like  the  terms  developmental  test 
(DT)  or  operational  test  (OT).  For 
that  matter,  I  don't  like  the  term 
integrated  test  either.  The  terms 
generally  describe  test  and 
evaluation  activities  at  different 
stages  of  capability  maturity,  but 
they  also  allude  to  the  different 
organizations  that  dictate  the 
terms  and  conditions  of  the 
test— the  program  manager 
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for  DT,  an  operational  test  agency  (OTA)  for  OT,  and  some 
combination  of  the  two  to  do  integrated  testing.  I  guess  the 
reason  I  don't  like  the  terms  is  because  they  represent  a  who 
does  what ,  when  model  for  T&E  instead  of  a  model  focused 
on  the  capability.  I  also  don't  like  the  terms  because  the  DT/ 
OT  model  is  not  complete— there  is  far  more  to  T&E  than 
DT  and  OT. 

I  believe  the  fundamental  purpose  of  T&E  is  to  enable  suc¬ 
cessful  acquisitions  of  enhanced  capabilities  for  the  war¬ 
fighter.  I've  chosen  those  words  carefully. 

T&E  is  an  enabling  process.  It  is  not  a  question  of  who  does 
what,  but  a  question  of  so  what?— that  is,  once  the  test  is 
done,  regardless  of  by  whom,  are  we  confident  that  the  new 
capability  will  improve  something  for  the  warfighters?  To  be 
an  enabler  in  acquisition,  we  need  a  model  for  T&E  that  is 
holistic,  in  which  every  test  event  is  a  shared  resource  of  all 
stakeholders,  regardless  of  when  it  occurs,  with  one  purpose 
in  mind:  To  answer  the  so  what  question.  Our  model  must 
de-emphasize  the  who  and  emphasize  the  what.  The  follow¬ 
ing  paragraphs  discuss  a  way  to  get  there  from  here. 

A  Rice  Bowl  Environment 

First,  we  need  to  understand  where  we  are  today.  Through 
the  course  of  evolution  of  the  DoD  5000,  we  have  created 
a  multitude  of  process  owners— materiel  developer,  com¬ 
bat  developer,  user,  tester,  decision  maker.  Today,  we  are 
also  thinking  about  capability  portfolio  managers,  although 
their  role  in  the  acquisition  process  has  yet  to  be  determined. 
Suffice  it  to  say  that  in  the  course  of  creating  the  acquisi¬ 
tion  process,  we  have  built  a  complex  environment  of  rice 
bowls  (meaning  a  person's  small  part  of  a  bigger  process) 
and  process  ownership.  And  in  some  cases,  process  owners 
staunchly  protect  their  rice  bowl. 

Moreover,  when  the  department  merged  acquisition  of  au¬ 
tomated  information  systems  into  DoD  5000  in  1996,  we 
added  more  process  owners,  such  as  the  interoperability 
certifier  and  the  designated  approving  authority  (informa¬ 
tion  assurance  certifier).  (From  1978  to  1996,  DoD  managed 
acquisition  of  AIS  under  DoD  7920  and  8120  directives  and 
instructions.  The  1996  issuance  of  the  DoD  5000  consoli¬ 
dated  weapons  and  AIS  acquisitions.)  However,  we  did  not 


fully  define  their  role  in  acquisition  decision  making.  For  ex¬ 
ample,  the  process  owners  for  interoperability  and  informa¬ 
tion  assurance,  the  Joint  Staff  J6,  and  DAA  respectively,  do 
not  sign  the  T&E  master  plan,  even  though  they  are  principal 
customers  of  significant  T&E  activities.  And  when  it  comes 
to  a  fielding  decision,  the  milestone  decision  authority  can 
make  a  decision  to  buy  capabilities  for  fielding  to  the  enter¬ 
prise,  but  the  DAA  can  deny  operations  of  that  capability 
on  the  local  network. 

The  traditional  approach  to  developing  a  T&E  strategy  for  an 
acquisition  program  is  to  knit  together  a  series  of  test  events 
that  we  generally  describe  as  either  DT  or  OT  (live  fire  T&E 
is  not  addressed  in  this  article).  In  doing  so,  we  tacitly  assign 
responsibility  for  those  events  to  their  respective  process 
owners— the  PM  plans  and  conducts  DT;  an  OTA  is  respon¬ 
sible  for  OT.  Somewhere  in  the  mix,  we  add  interoperability 
and  information  assurance  test  events,  and  responsibility 
for  those  activities  is  thereafter  delegated  to  their  process 
owners.  Once  the  many  parties  agree  to  the  strategy,  the 
process  owners  move  off  to  their  respective  corners  and  plan 
their  events,  and  coordination  between  them  is  minimal  if  it 
occurs  at  all.  This  is  a  worst-case  scenario,  of  course;  not  all 
programs  experience  this. 

Recent  policy  revisions  attempt  to  influence  and  improve 
the  coordination  between  the  process  owners,  by  blending 
DT  and  OT  into  an  integrated  testing  model  that  is  seam¬ 
less  throughout  a  system's  life  cycle  (see  Memorandum, 
Subject:  Test  and  Evaluation  Policy  Revisions,  DOT&E  and 
AT&L,  Dec.  22, 2007).  The  new  policy  does  not  specifically 
identify  interoperability  testing  and  information  assurance 
as  part  of  the  integrated  test  model,  but  an  integrated  model 
is  not  complete  without  them.  At  its  core,  however,  inte¬ 
grated  testing  is  fundamentally  a  call  for  early  involvement 
to  bring  the  government's  testers  forward  in  the  acquisition 
process.  In  the  words  of  the  new  policy,  “T&E  expertise  must 
be  brought  to  bear  at  the  beginningof  the  system  life  cycle...'' 
This  is  based  on  the  theory  that  early  involvement  of  the 
testers  leads  to  early  problem  discovery  and  correction,  and 
therefore  the  program  is  more  likely  to  successfully  negotiate 
the  acquisition  process  and  achieve  a  fielding  decision. 

Early  involvement  has  been  a  consistent  theme  in  T&E  in  the 
department  for  decades.  So  why  is  it  so  hard  to 
come  by?  The  answer  is  a  bit  of  a  blinding  flash  of 
the  obvious:  Because  we  made  it  this  way. 

The  Myth  of  Early  Involvement 

There  is  a  saying  that  a  picture  is  worth  a  thou¬ 
sand  words.  Unfortunately,  even  though  I'm 
using  pictures,  this  paper  will  not  be  thousands 
of  words  shorter. 

Figure  1  is  a  picture  of  the  Defense  Acquisition 
Management  Framework  taken  from  the  DoD  In¬ 
struction  5000.2.  Observe  how  the  graphic  con- 


Figure  1:  The  Defense  Acquisition  Management  Framework 
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Figure  2:  IOT&E  in  the  Acquisition  Process 


veys  the  relationship  of  T&E  to  the  acquisition  process:  one 
test— the  initial  operational  T&E  (IOT&E)— post  Milestone 
C!  This  is  not  a  very  complete  picture  of  the  role  of  T&E,  and 
certainly  not  one  that  depicts  early  involvement. 

An  equally  familiar  and  far  more  detailed  view  of  the  acqui¬ 
sition  process  can  be  seen  in  the  “Integrated  Defense  Ac¬ 
quisition,  Technology,  and  Logistics  Life  Cycle  Management 
Framework  Chart,"  or  more  commonly  called  the  "wall  chart." 
(Go  to  <https://acc.dau.mil/IFC/index.htm>  for  a  complete 
view  of  the  wall  chart.)  Finding  T&E  in  this  wall  chart  is  a  bit  like 
looking  at  a  Where's  Waldo?  picture  book.  The  one  test  event 
that  is  so  prominent  in  the  first  figure  is  here  as  well;  it's  just 
hard  to  find.  Figure  2  zooms  in  to  show  the  IOT&E. 

Now,  given  this  greater  detail,  look  at  how  we  illustrate  the 
IOT&E  in  the  acquisition  process:  the  output  feeds  a  critical 
report,  which  in  turn  feeds  a  decision  event,  but— do  you  see 
it?— there  are  no  inputs! 


The  thousand  words  described  by  these  pictures 
can  be  summarized  in  these  few:  Testers  are  not 
involved  early,  and  what  happens  in  the  develop¬ 
ment  phase  has  no  bearing  on  the  IOT&E.  Note  that 
it  says  the  same  thing  about  interoperability  testing 
as  well.  That  is,  of  course,  not  the  way  it  is  in  the  real 
world;  I'm  just  trying  to  shed  light  on  the  myth  of 
early  involvement. 

A  closer  inspection  of  the  wall  chart,  however,  reveals 
seven  different  T&E  activities  (not  including  live  fire 
T&E  or  the  military  utility  assessment  associated  with 
the  advanced  concept  technology  demonstrations). 
Figure  3  zooms  in  to  show  these  seven  activities.  Ob¬ 
serve  that  T&E  activities  do  not  begin  until  the  latter 
part  of  the  system  demonstration  phase— again,  not  what  we 
consider  early  involvement. 

Our  pictures  need  to  tell  a  different  story.  More  importantly, 
our  DoD  directives  and  instructions  need  to  tell  a  different 
story. 

The  Reality  of  Early  Involvement 

If  the  measure  of  our  early  involvement  were  the  number  of 
programs  found  effective  and  suitable  today,  I'd  say  we've 
been  found  wanting  as  an  enabler  of  successful  acquisi¬ 
tions. 

There  is  a  very  basic  explanation  for  why  we  have  such  trou¬ 
ble  with  early  involvement  and  integrated  testing:  Because 
we  don't  have  to.  The  DoDI  5000.2  creates  these  rice  bowls 
and  assigns  their  process  owners.  For  example,  in  the  May 
2003  version  of  5000.2,  paragraph  E5.1.2  says,  "The  PM 
shall  design  DT&E  objectives  appropriate  to  each  phase  and 
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We  wrote  an  acquisition  model 
that  fosters  an  environment 
of  process  owners  who 
protect  rice  bowls.  But 
since  we  wrote  the 


tenet  of  integrated  testing  is  to  get  the  OTA  involved  in  DT, 
so  we  still  have  not  accomplished  the  change  that  is  needed. 
A  little  further  in  Enclosure  5  (May  2003  version)  are  the 
paragraphs  E5.1.5and  E5.1.7,  which  have  the  following  para¬ 
graph  headers: 

■  "E5.1.5  Developmental  Test  and  Evaluation  (DT&E). 
During  DT&E,  the  materiel  developer  shall ..." 

■  "E5.1.7  Operational  Test  and  Evaluation  (OT&E)." 


model,  we  can 
rewrite  it. 


milestone  of  an  acquisition  program. ...  The  OTA  shall  design 
OT&E  objectives ..." 


Again  we  see  the  5000  delineating  responsibility,  especially  in 
the  case  of  DT.  Reading  through  the  sub-paragraphs,  it  is  clear 
that  integrated  testing  is  neither  expected  nor  encouraged. 
Nowhere  within  E5.1.5  or  E5.1.7  are  instructions  requiring  co¬ 
ordination  between  the  materiel  developer  and  the  OTA.  In¬ 
terestingly,  subparagraph  E5.1.5.8.  does  instruct  the  materiel 
developerto  "support  the  DoD  Information  Technology  Secu¬ 
rity  Certification  and  Accreditation  Process  [DITSCAP]  and 
Joint  Interoperability  Certification  process"  during  DT&E. 


Those  statements  make  it  clear  who  owns  DT  and  who  owns 
OT.  A  subtle  change  occurred,  however,  in  the  revisions  cur¬ 
rently  proposed  to  the  5000.02.  At  the  time  of  this  writing, 
the  final  draft  of  the  5000.02,  paragraph  E5.3,  uses  the 
following  wording:  "The  PM  shall  design  DT&E  objectives 
appropriate  to  each  phase  and  milestone  of  an  acquisition 
program. ...  The  OTA  and  the  PM  shall  collaboratively  design 
OT&E  objectives  [emphasis  added] ... " 

Isn't  it  interesting  that  collaboration  between  the  PM  and 
OTA  is  indicated  for  OT&E,  but  not  for  DT&E?  The  main 


The  new  version  of  the  5000.02  is  fundamentally  unchanged 
with  regard  to  the  content  of  these  paragraphs.  That's  disap¬ 
pointing,  especially  given  the  recent  emphasis  on  integrated 
testing.  One  might  have  expected  instructions  for  materiel 
developers  to  consider  OTA  input  in  developmental  test 
designs  and  allowing  OTAs  to  collect  data  during  DT.  At 
the  extreme,  one  might  expect  to  eliminate  the  paragraphs 
on  DT  and  OT  altogether  and  substitute  them  with  a  single 
paragraph  on  integrated  testing. 

That's  the  blinding  flash  of  the  obvious— we  wrote  an  ac¬ 
quisition  model  that  fosters  an 
environment  of  process  owners 
who  protect  rice  bowls.  But  since 
we  wrote  the  model,  we  can  re¬ 
write  it. 

Different  Terminology 

We  have  a  lot  of  different  terms 
for  the  types  of  T&E  we  do,  but 
not  many  of  them  have  universally 
accepted  definitions.  Depending 
on  where  you  look,  you  can  find 
different  definitions  for  most  of 
our  common  terms.  For  example, 
even  the  term  operational  testing, 
which  has  the  widely  accepted 
definition  given  in  Title  10,  §139, 
differs  in  Joint  Publication  1-02, 
the  Department  of  Defense  Dic¬ 
tionary  of  Military  and  Associated 
Terms. 

A  quick  check  of  the  Glossary  of 
Defense  Acquisition  Terms  and  the 
Test  and  Evaluation  Management 
Guidebook  shows  that  some  of 


Figure  4:  Test  and  Evaluation  in  the  DoD  Acquisition  process 
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our  most  common  T&E  terms— like  DT,  IOT&E,  and  follow- 
on  OT&E— are  all  defined  differently  to  some  degree.  And 
despite  the  fact  that  we  have  been  talking  about  integrated 
testing  for  decades,  neither  of  those  sources  provide  a  defi¬ 
nition  of  the  term. 

On  April  25,  2008,  the  director  for  operational  test  and 
evaluation  and  the  deputy  under  secretary  of  defense  for  ac¬ 
quisition  and  technology  provided  a  definition  of  integrated 
testing:  “Integrated  testing  is  the  collaborative  planning  and 
collaborative  execution  of  test  phases  and  events  to  provide 
shared  data  in  support  of  independent  analysis,  evaluation, 
and  reporting  by  all  stakeholders,  particularly  the  develop¬ 
mental  (both  contractor  and  government)  and  operational 
test  and  evaluation  communities." 

There  are  three  key  elements  to  this  definition:  collabora¬ 
tion,  shared  data,  and  involvement  of  all  stakeholders.  For 
most  systems  in  the  acquisition  pipeline  today,  there  is  an 
information  technology  element,  be  it  software,  hardware,  or 
communications.  Integrated  testing  gets  harder  when  T&E 
for  information  technology  is  part  of  the  equation.  When  do 
we  do  the  information  assurance  and  interoperability  tests? 
Under  what  conditions?  Who  can  do  the  testing?  Who  is 
the  customer?  Organizations  other  than  the  PM  and  opera¬ 
tional  test  authority  may  have  to  be  brought  in  to  perform 
the  tests:  the  Joint  Interoperability  Test  Command  for  the 
joint  interoperability  certification;  and  for  the  information 
assurance  certification,  the  program  might  have  to  bring  in 
a  security  tester,  such  as  the  National  Security  Agency  or 
Defense  Intelligence  Agency. 

Note  the  use  of  terminology  on  the  acquisition  wall  chart: 

■  Individual  Cl  [ Configuration  Item']  Verification  DT&E 

■  Integrated  DT&E,  LFT&E  [live  fire  test  and  evaluation ], 
and  EOAs  [early  operational  assessment] 

•  System  DT&E,  LFT&E,  and  OAs 

•  Combined  DT&E/OT&E/ LFT&E 

■  Independent  IOT&E 

■  JITC  Interoperability  Certification  Testing 

•  FOT&E. 

It  is  hard  to  find  definitions  of  all  of  these  terms.  However, 
an  important  characteristic  of  the  terms  is  that  they  reflect 
a  progression  from  testing  individual  components  to  the  in¬ 
tegrated  system,  as  well  as  increasing  operational  realism— 
from  EOAs  to  OAs  to  OT&E.  That  type  of  progression  is  a 
good  thing,  but  with  all  this  emphasis  on  integrated  testing, 
the  terminology  might  need  work.  And  of  course,  as  de¬ 
picted  on  the  wall  chart,  T&E  starts  late  in  the  game— early 
involvement  should  move  most  of  the  T&E  activities  shown 
in  Figure  4  into  technology  development  and  system  integra¬ 
tion.  Also,  the  way  the  picture  tells  it,  interoperability  testing 
is  not  part  of  the  integrated  test  model,  and  it's  noteworthy 
that  information  assurance  certification  testing  is  not  on  the 
chart  (there  are  references  to  DITSCAP  certifications  on  the 
back  of  the  wall  chart;  the  Defense  Information  Assurance 


Certification  and  Accreditation  Program  has  since  replaced 
the  DITSCAP).  We  need  a  complete  picture. 

The  question  is  how  to  create  a  framework  for  T&E  in  DoD 
that  combines  all  of  the  elements  described  above  into  a 
more  efficient  and  effective  process.  I  propose  a  new  model 
that  will  do  that— I  call  it  the  Capability  Test  and  Evaluation 
Model. 

Capability  Test  and  Evaluation  Model 

A  common  trend  in  DoD  is  to  talk  in  terms  of  capabilities:  the 
term  requirements  is  out,  and  capabilities  is  in;  the  term  threat- 
based  is  out,  and  capability-based  is  in.  Moreover,  we  now 
hear  about  capability  portfolios  and  joint  capability  areas. 
Hence  the  name  Capability  Test  and  Evaluation,  or  CT&E. 

The  intent  of  the  CT&E  Model  is  to: 

■  Share  information 

■  Improve  risk  management 

■  Eliminate  duplication  and  reduce  cost 

■  Conduct  comprehensive,  mission-focused  test  events, 
faster 

■  Ensure  decision  makers  and  users  have  all  relevant 
information  to  better  understand  capabilities  and  limita¬ 
tions. 

In  other  words,  the  intent  of  CT&E  is  to  enable  rapid  acquisi¬ 
tion  of  enhanced  capabilities  for  the  warfighter. 

We  must  recognize  that  T&E  is  a  continuous  process 
throughout  the  program  life  cycle,  not  just  one  event  oc¬ 
curring  after  Milestone  C.  Multiple  process  owners  conduct 
T&E.  However,  because  we  do  not  have  one  organization 
that  is  ultimately  in  charge  of  all  of  these  T&E  activities,  we 
foster  an  environment  of  serial  events,  multiple  reports,  and 
incomplete  information  to  decision  makers. 

Capability  T&E  is  all  about  unity  of  effort.  But  to  achieve  this 
unity  of  effort,  we  need  unity  of  command— a  good  military 
phrase  meaning  somebody  has  to  be  in  charge.  There  are  at 
least  four  different  test/certification  activities  on  the  road  to 
a  fielding  decision— different  tests,  for  different  customers, 
conducted  under  different  conditions,  and  under  different 
rules.  Figure  4  depicts  the  relevant  T&E  and  certification 
activities  that  occur  in  the  acquisition  process. 

Capability  T&E  brings  the  four  test/certification  activities 
into  each  test  event,  beginning  as  early  in  the  acquisition 
process  as  practical.  The  CT&E  model  can  therefore  be  de¬ 
scribed  as  one  team,  one  set  of  conditions,  every  time.  The 
objective  of  CT&E  is  to  satisfy  the  decision-making  needs 
of  all  test  customers.  CT&E  test  designs  are  risk-based,  mis¬ 
sion-focused.  Typical  users  exercise  the  capability  during  the 
test.  A  capability  test  team  plans  and  conducts  the  CT&E 
Model,  and  ideally,  prepares  one  report  for  submission  to  all 
customers.  CT&E  in  no  way  limits  the  independence  of  the 
OTA  or  its  ability  to  provide  independent,  objective  evalua- 
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FROM  OUR  READERS 


Some  Additional  Rules 

I  liked  Wayne  Turk's  article  “Step  up  to  the  Podium" 
in  the  September-October  2008  issue  of  Defense 
AT&L  magazine.  It  presented  many  practical  tips 
for  preparing,  crafting  and  giving  an  effective  pre¬ 
sentation,  and  preventing  the  dreaded  "Power¬ 
Point®  Poisoning"  that  is  so  common  these  days.  I 
plan  to  distribute  the  article  to  all  the  members  in 
my  division  as  a  guide  for  when  they  need  to  make 
a  presentation. 

I  would  like  to  suggest  another  technique  for  effec¬ 
tive  presentations.  A  lot  of  benefit  can  be  realized 
with  pre-briefs  of  meeting  participants  before  the 
actual  presentation  is  given.  Pre-briefs  and  offline 
meetings  allow  a  lot  of  peer  review  prior  to  the 
formal  presentation.  It's  a  good  opportunity  to  get 
early  feedback  to  be  able  to  tweak  the  presentation 
and  avoid  dropping  any  bombshells  at  the  actual 
meeting.  We  do  this  routinely  here  at  Naval  Air 
Systems  Command.  A  pre-brief  also  allows  people 
to  concentrate  more  fully  at  the  actual  presentation 
because  it's  not  the  first  time  they've  seen  it  and 
they  don't  have  to  so  many  questions. 

I  also  liked  Brian  J.  Duddy's  article  "To  Boldly  Go ... 
Into  Defense  Acquisition:  The  Program  Manager's 
Rules  Of  Acquisition"  in  the  September-October 
2008  issue  of  Defense  AT&L  magazine.  The  Star 
Trek  theme  was  an  entertaining  way  to  effectively 
present  important  information.  I  liked  the  rules  the 
author  cited,  especially  the  ones  about  clarity  in  the 
statement  of  work.  And  I  agree  whole-heartedly 
that  verbal  agreements  aren't  enough. 

I  would  like  to  suggest  that  formal  contract  modifi¬ 
cations  aren't  always  necessary.  Naval  Air  Systems 
Command  routinely  holds  technical  interchange 
meetings,  and  the  minutes  from  these  meetings 
provide  the  written  agreements  about  changes  that 
are  made.  Minutes  are  rarely,  if  ever,  disputed,  and 
are  a  much  easier,  cheaper,  and  faster  mechanism 
than  a  formal  contract  modification  to  document 
changes.  Also,  making  every  agreement  a  contract 
modification  can  present  a  significant  workload  in¬ 
crease  for  our  contracts  department.  We  usually 
reserve  contract  modifications  for  when  there  is  a 
change  that  involves  money  or  a  change  in  scope 
of  the  contract. 

Al  Kaniss 

Naval  Air  Systems  Command 


T&E  is  an  enabling  process.  It 
is  not  a  question  of  who  does 
what,  but  a  question  of  so 
what?— that  is,  once  the  test  is 
done  ...  are  we  confident  that 
the  new  capability  will  improve 
something  for  the  warfighters? 


tions  of  capability  effectiveness  and  suitability.  The  condi¬ 
tions  for  test  should  replicate  the  joint  mission  environment 
and  leverage  distributed  live,  virtual,  and  constructive  T&E 
capabilities  to  the  maximum  extent  possible. 

The  Defense  Acquisition  Guidebook  says  that  the  milestone 
decision  authority  should  designate  the  lead  operational  test 
agency  to  coordinate  all  operational  test  and  evaluation.  The 
lead  operational  test  agency  should  produce  a  single  opera¬ 
tional  effectiveness  and  suitability  report  for  the  program. 
(DAG,  paragraph  11.1.2.2.) 

Let's  change  the  DAG  to  read,  "The  milestone  decision  au¬ 
thority  should  designate  a  responsible  test  organization  to 
coordinate  all  test,  evaluation,  and  certification  activities. 
At  the  conclusion  of  each  test  activity,  the  responsible  test 
organization  should  produce  a  single  capability  evaluation 
report  for  submission  to  the  MDA,  the  Joint  Staff  (for  in¬ 
teroperability  certification),  and  the  DAA  (for  information 
assurance  certification)." 

In  the  next  round  of  updates  to  the  DoD  5000,  let's  eliminate 
the  rice  bowls  and  focus  on  the  capability  being  proposed 
for  fielding  to  our  warfighters. 

Making  Integrated  Testing  a  Reality 

Every  test  event  should  be  considered  a  shared  resource. 
Integrated  testing  is  not  just  about  early  involvement;  it's 
about  sharing  information  to  improve  our  understanding 
of  capabilities  and  limitations.  As  a  shared  resource,  every 
stakeholder  should  have  some  say  in  how  the  event  is  con¬ 
structed  so  it  satisfies  some  part  of  their  needs.  To  be  suc¬ 
cessful  at  integrated  testing  will  require  some  non-traditional 
thinking  and  the  breaking  of  those  rice  bowls.  Moreover,  in¬ 
tegrated  testing  is  not  just  a  matter  of  saying  it;  we  have  to 
teach  it,  train  it,  demand  it,  plan  it,  and  practice  it.  So  let's 
get  on  with  it. 


The  author  welcomes  comments  and  questions  and  can  be 
contacted  at  steven. hutchison@disa. mil. 
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